Temporal and Information Flow Based Event Detection from Social Text Streams

Recently, social text streams (e.g., blogs, web forums, and emails) have become ubiquitous with the evolution of the web. In some sense, social text streams are sensors of the real world. Often, it is desirable to extract real world events from the social text streams. However, existing event detection research mainly focused only on the stream properties of social text streams but ignored the contextual, temporal, and social information embedded in the streams. In this paper, we propose to detect events from social text streams by exploring the content as well as the temporal, and social dimensions. We define the term event as the information flow between a group of social actors on a specific topic over a certain time period. We represent social text streams as multi-graphs, where each node represents a social actor and each edge represents the information flow between two actors. The content and temporal associations within the flow of information are embedded in the corresponding edge. Events are detected by combining text-based clustering, temporal segmentation, and information flow-based graph cuts of the dual graph of the social networks. Experiments conducted with the Enron email dataset and the political blog dataset from Dailykos show the proposed event detection approach outperforms the other alternatives. Introduction Recently, social text stream data, such as weblogs, message boards, mailing lists, review sites and web forums, have become ubiquitous with the evolution of the web. We refer to a collection of text communication data that arrives over time as social text stream data, where each piece of text in the stream is associated with some social attributes such as author, reviewer, sender, and recipients. In the last few years, social text stream data has changed the way we communicate daily, the way we market businesses, and even the way political campaigns are conducted. For example, weblogs are now used not only by individual users for sharing information about their daily lives and thoughts, but are also used by large business corporations and political parties to Copyright c © 2007, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. 1http://www.cs.cmu.edu/∼enron/ http://www.dailykos.com release their latest products or new proposals. Usually, social text stream data arrives over time and each piece of the stream carries part of the semantics (e.g., information about real world events) (Kleinberg 2006). In this sense, social text streams are sensors of the real world. Social text streams generate large amounts of text data from various types of sources. The streams have rich content such as text, social networks, and temporal information. Efficiently organizing and summarizing the embedded semantics has become an important issue. Most text semantic analysis techniques mainly focused on TDT (Topic Detection and Tracking) (Krause, Leskovec, & Guestrin 2006; Yang, Pierce, & Carbonell 1998) for general text data (e.g., scientific papers, newswires, and corporation documents). However, the social text stream data is substantially different from general text stream data: (1) social text stream data contains rich social connections (between the information senders/authors and recipients/reviewers) and temporal attributes of each text piece; and (2) the content of text piece in the social text stream data is more context sensitive. That is, not only within the text piece content, the meaning of words are dependent from the context; but also the meaning of a text piece is dependent from the social actors (e.g., sender, author, recipient, and commenter) and other temporally correlated text pieces (e.g., the temporal and content information of previous text pieces). In this work, we propose to detect events from social text streams by exploring features in three dimensions: textual content, social, and temporal. In the literature, there are some existing works on semantic analysis of text stream data and social network data (McCallum & Huang 2004; Mei et al. 2006; Yang, Pierce, & Carbonell 1998). For example, different text content analysis techniques have been proposed to classify huge amount of emails into different topics or identify entities (McCallum & Huang 2004). Event detection algorithms have been proposed for newswires (Yang, Pierce, & Carbonell 1998) and blogs (Mei et al. 2006). Different communities can be extracted from the email communication network as well (Leuski 2004). However, most of them ignored either the social network information, or the temporal properties of the stream data. However, the embedded temporal and information flow pattern (communication pattern), and the social network relations in social text stream data can be used to improve the